GIVersa-Endoscopy: A Large Language Model-Based AI Assistant for Endoscopy Sedation Triage

Introduction:

Sedation planning before endoscopic procedures is essential to ensure patient safety and appropriate allocation of anesthesia resources.^1,2 At many institutions, this process relies on manual chart review by clinical staff and endoscopists. For direct endoscopy referrals at our institution, clinicians determine sedation modality based on patient comorbidities and institutional criteria. This requires review of more than 150 referrals weekly and contributes to administrative burden. As health systems expand across broader geographic regions and multiple endoscopy sites, triaging patients to the safest procedural setting becomes increasingly complex.

Large language models (LLMs), a form of generative artificial intelligence (GenAI), have demonstrated capabilities in clinical information processing tasks including risk stratification, data extraction, and summarization.^3–6 These tools may help streamline workflows and support decisions. In this study, we developed an LLM-based assistant, “GIVersa-Endoscopy,” to support sedation triage for patients referred for upper endoscopy and colonoscopy and evaluated its performance compared with clinician triage decisions.

Methods:

Detailed methods are provided in the Supplementary Materials. In brief, we customized our institution’s protected health information (PHI)-compliant, Microsoft Azure OpenAI GPT-4o-based LLM, Versa, using the American Society of Gastrointestinal Endoscopy and our institutional endoscopy sedation guidelines via retrieval-augmented generation (RAG).^7–9 This chat-based GenAI assistant, “GIVersa-Endoscopy,” combines free-text and structured electronic health record (EHR) data to recommend for patients referred for endoscopy (1) sedation type (moderate sedation versus anesthesia support, defined as at least monitored anesthesia care) and (2) endoscopy center location as a surrogate for anesthesia risk level.

We retrospectively evaluated 95 randomly selected patients referred for upper endoscopy or colonoscopy between August and December 2024. Structured data were extracted into a SmartPhrase free-text note and supplied directly to the GenAI assistant without modification. Agreement was defined as concordance between the GenAI assistant and clinician decisions. Performance was assessed using precision, recall, specificity, and F1 score with 95% confidence intervals (CI).

Results:

The 95 patients had a mean age of 60 ± 15 years and 52 (55%) were female. Most identified as White Non-Hispanic or Latino (50.5%). Primary endoscopy indications were screening (33.7%) and surveillance (28.4%). Comorbidities included body mass index > 40 (7.4%), obstructive sleep apnea (17.9%), major cardiac disease (31%), pulmonary disease (11%), stroke or seizure (7%), substance use (66%), chronic kidney disease (10.5%), and cirrhosis (2.3%).

GIVersa-Endoscopy matched clinician sedation recommendations in 75% (71/95) of cases and triaged at the same level or more conservatively in 91%. Among the 24 discordant cases (25%), 8 (35%) lacked sufficient EHR documentation, 6 (26%) involved complex clinical scenarios not addressed by guidelines, 3 (9%) reflected model misinterpretation of clinical data, and 7 (30%) reflected additional clinician context. Documentation-related discordance included outdated problem lists or weights. Model misinterpretation included failure to recognize that a kidney transplant patient was no longer on dialysis. Cases reflecting additional clinician context included interpretation of stable cardiac histories or electrocardiographic findings (n=3), clarification of psychosis severity not captured in the simplified EHR (n=1), and contextual judgement such as anticipating higher anesthesia needs for a young patient (n=2).

For identifying cases requiring anesthesia support, precision was 76%, recall 84%, specificity 61%, and F1 score 80% (95% CI 0.72–0.88). For moderate sedation, precision was 72%, recall 61%, specificity 84%, and F1 score 66% (95% CI 0.52–0.78). For cases in which sedation type matched clinician recommendations, the assistant correctly triaged endoscopy center location as a surrogate for anesthesia risk level 100% of the time (F1 score 1.0; 95% CI 0.93–1.00). (Table 1)

Table 1:

Performance of GIVersa-Endoscopy in the retrospective evaluation of 95 patients directly referred for endoscopy: (a) confusion matrix for sedation type triage and (b) performance metrics. Agreement was defined as concordance between the GenAI assistant and clinician triage decisions. In the confusion matrix, true positive represent cases in which both clinician and assistant recommended anesthesia support and true negatives represent cases in which both recommended moderate sedation. False positives represent conservative over-triage (assistant recommending anesthesia support when clinician recommended moderate sedation), and false negatives represent liberal under-triage (assistant recommending moderate sedation when the clinician recommended anesthesia support).

a. Confusion Matrix for Sedation Type Triage
	Predicted: Anesthesia Support	Predicted: Moderate Sedation
Clinician: Anesthesia Support	48	9
Clinician: Moderate Sedation	15	23

b. Performance Metrics
Task	Class	Precision	Recall	Specificity	F1 score	95% CI
Sedation Type	Anesthesia	0.76	0.84	0.61	0.80	0.72–0.88
	Moderate	0.72	0.61	0.84	0.66	0.52–0.78
Location/risk level*	Overall	1.00	1.00	1.00	1.00	0.93–1.00**

^*Anesthesia risk level triage was conditionally evaluated only when sedation type matched.

^**Binomial confidence interval for components rather than bootstrap CI for a F1 score of 1.

Discussion:

This proof-of-concept study describes the development of GIVersa-Endoscopy, a customized RAG-based LLM to augment sedation triage decisions for routine endoscopy. In this retrospective evaluation, the assistant achieved 75% agreement with clinician decisions regarding sedation type and demonstrated stronger performance in identifying patients requiring anesthesia support (F1 0.8) than those appropriate for moderate sedation. The model’s F1 score of 1.0 for the second triage task of endoscopy location as a proxy for anesthesia risk level was evaluated in a small sample and only when sedation type matched, reflecting task dependency rather than independent predictive ability. Nevertheless, performance across anesthesia support locations suggests potential for more nuanced triage tasks.

Most discordant cases were attributable to insufficient documentation, ambiguous guideline coverage, or additional clinician context rather than intrinsic model failure. In cases of disagreement, the assistant more frequently recommended higher anesthesia support, reflecting a conservative tendency. From a patient perspective, the highest risk scenario would be sedation complications without appropriate anesthesia support. However, over-triage may introduce system-level consequences including increased cost, scheduling delays, reduced access to limited anesthesia resources, or unnecessary anesthesia exposure. Our retrospective design did not allow assessment of these downstream effects. Therefore, we interpreted the model’s conservative behavior within a patient safety framework as an initial step toward building trust and accountability in GenAI-assisted clinical workflows.¹⁰

Several other limitations should be acknowledged. First, clinician triage decisions served as the reference standard. Sedation triage is inherently subjective and may vary by provider experience and risk tolerance. Accordingly, our results reflect agreement with clinician judgment rather than accuracy, as we did not validate against an independent gold standard such as anesthesia-related outcomes or adverse events. Future prospective evaluation will incorporate blinded physician experts and post-procedure outcomes. Second, institutional AI governance precluded direct integration with the EHR, requiring a proxy pipeline using SmartPhrase free-text notes derived from structured EHR elements. Although designed to approximate real-world documentation, this approach may not capture all nuances present in full chart review. Third, run-to-run variability was not formally assessed. Because LLM outputs are probabilistic, temperature was set to 0 and each case was evaluated in an independent session with cleared model memory. Finally, GIVersa-Endoscopy was developed within a single institution using a local GenAI platform. While performance may not directly generalize across environments, the framework is adaptable to other institutional EHR infrastructures and GenAI platforms.

In summary, GIVersa-Endoscopy demonstrates the feasibility of a PHI-protected, RAG-based LLM model to support sedation triage in gastroenterology. Prospective evaluation with direct workflow integration will be necessary to assess clinical impact, reproducibility, clinician acceptance, and associations with patient outcomes before broader implementation.

What You Need to Know.

Background:

Sedation triage in endoscopy is essential for patient safety and anesthesia resource optimization. Current triage relies on time-consuming manual review thereby providing an opportunity to use generative AI tools to augment clinical decision support and automate clinical workflows.

Findings:

GIVersa-Endoscopy is a custom generative AI model that is accurate and conservative with respect to patient safety in sedation triage when compared to human clinicians in 91% of retrospective cases reviewed.

Implications for patient care:

Our endoscopy sedation triage AI assistant serves as a proof-of-concept illustrating the broader role of generative AI in enhancing periprocedural workflows and clinical decision support.

Abbreviations:

AI

Artificial intelligence

CI

confidence interval

EHR

electronic health record

GenAI

Generative artificial intelligence

LLM

large language model

PHI

protected health information

RAG

retrieval-augmented generation

Data Transparency Statement:

The data that support the findings of this study are not publicly available due to patient privacy and institutional restrictions. The analytic methods and study materials are available from the corresponding author upon reasonable request.